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ABSTRACT 



This paper reports the results of a study of the development 
of an innovative learning element designed to be implemented in a 
computer- facilitated learning (CFL) module. The learning element is an 
open-ended, short answer, text question tool (TQT) designed to be used in 
World wide Web-based courses or incorporated into hybrid Web/CD-ROM systems. 
The TQT object facilitates the development of extended question-and-answer 
problems which overcome the limitations of multiple choice question format. 
The TQT facilitates an iterative approach to developing student knowledge 
constructions. The examples provided in this paper relate to questions 
developed for the Department of Anatomy, Faculty of Medicine at the 
University of Melbourne (Australia) . However, it is stressed that the TQT is 
applicable to many other academic disciplines with similar needs to develop 
student understanding of specific content domains. Topics addressed include 
functional aspects of the TQT, the TQT and student learning, and current 
development and evaluation. Three figures illustrate the structural view of 
the TQT, a functional view of the TQT object, and the prototype of the TQT; 
one table presents an example proforma for anatomy questions. (Author/DLS) 
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Abstract? This paper reports the results of a study of the development of an 
innovative learning element designed to be implemented in a computer-facilitated 
learning (CFL) module. The learning element is an open-ended, short answer, text 
question tool (TQT) designed to be used in Web-based courses or incorporated into 
hybrid Web/CD-ROM systems. The TQT object facilitates the development of 
extended question-and-answer problems which overcome the limitations of the 
multiple choice question format. The TQT facilitates an iterative approach to 
developing student knowledge constructions of knowledge. The examples provided 
in this paper relate to questions developed for the Department of Anatomy, Faculty 
of Medicine at The University of Melbourne. However, it must be stressed that the 
TQT is applicable to many other academic disciplines with similar needs to develop 
student understanding of specific content domains. 



Introduction 

The development of the Text Question Tool (TQT) is an additional component of the Learning Engines 
project — work being undertaken by the Multimedia Education Unit (MEU) at The University of Melbourne to 
develop simple Web-based learning tools [Fritze & McTigue 1997]. The project is developing a number of 
generic learning tools for the implementation of Web-based, CD-ROM or Web/CD-ROM hybrid course 
materials. A number of these tools have already been produced including an a concept mapping tool, tutorial 
item set tool, and an interactive graphing tool [Kennedy, Fritze, & McTigue, 1997]. The Text Question Tool is 
designed to be a generic question tool which facilitates the construction of computer-facilitated learning (CFL) 
modules or Web-based courses. The functional aspects of the TQT are the: 

• facility to allow students to type in short text-based answers in a field in a Web page based upon a question 
item; 

• ability of the TQT to search for key words or phrases defined by the lecturer; 

• ability to provide feedback to students based upon the number of key words or phrases found in the text 
string typed in by the student; and 

• facility to respond to student requests for Hints, spelling of terms, and expert answers to the question. 

Faculty members from the departments of Chemistry, Physics, Anatomy, Pharmacology and Economics at 

The University of Melbourne were surveyed to determine the needs of delivering on-line tutorials. While the 
needs of each academic discipline were varied, the TQT was perceived to be fundamental to the implementation 
of Web-based courses. The department of Anatomy is participating in the development, design and the 
formative evaluation of the TQT. The design of cognitive tools for student learning is an outcome of a 
theoretical framework developed to link what is known about sound educational practice with the design of 
interactive elements to incorporate into multimedia courseware [Kennedy, Fritze, & McTigue, 1997; Kennedy & 
McNaught, 1997]. 
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The TQT fulfils a number of educational requirements not available in current Web-based courseware tools. 
These are: 

• allowing a student to express more complex concepts and ideas in her or his own words and receive 
immediate feedback in a Web-based environment; 

• providing an environment in which students can refine and reflect upon their understanding of concepts 
which is non-threatening; 

• allowing the content expert to decide the content matter and the appropriate form and language of the answer. 
It is the lecturer who decides the key words, hints and expert answers pertinent to a particular content 
domain; and 

• providing an opportunity for students to learn in an environment which is supported by a sound educational 
framework. 

A non- functional example of the look-and-feel of the TQT object with examples of questions may be found 
at <http://www3.meu.unimelb.edu.au/meuweb/staff/dkennedy/TQT.html>. The TQT has been through a number 
of iterations in discussions with academic staff from the department of Anatomy using the scenario-based 
approach. Scenarios have been chosen because they “are grounded in the work activities of uol/io, 

the work users do drives the development of the system intended to augment this work. Thus scenarios are often 
open-ended and fragmentary; they help developers and users pose new questions, question new answers, open up 
possibilities” [Carroll 1995], p. 5. 

Example of a scenario in Anatomy: The student reads a question — ^in conjunction with an range of resources 
(e.g., an series of anatomical images, a video clip of the initial phases of the proceedure, or images of the 
radiological perspective) from the lower back in the Interactive Anatomy (LA) database — and is required to link 
the procedure with the underlying anatomical structures. For example, the insertion of a needle for a lumbar 
puncture requires an understanding of the anatomical structures the needle must pass through, the angle of 
insertion, and how far the needle must be inserted in order to obtain cerebrospinal fluid. The student is required 
to apply her or his anatomical knowledge in order to solve a clinical problem — which determines the success or 
failure of this procedure. 

Teaching and learning 

The advent of the World Wide Web (Web) and the shift to delivery of courses via the Web has provided the 
impetus to develop pedagogically sound learning tools for student learning. It has become clear that merely 
placing lecture notes on the Web is not sufficient for student learning to occur. However, there is a lack of 
suitable interactive learning tools which are Web compatible to actively engage the learner. A major purpose of 
any learning tool should be to provide teachers and lecturers with the facility to match the desired educational 
outcomes of a Web-based course with the learning elements which have the greatest potential to achieve those 
outcomes. For example, studies from many institutions over a period of many years “have drawn attention to the 
wide gap between the rhetoric describing the qualities lecturers say they want in their students’ responses, and 
the tasks they set” [Biggs 1989], p. 15 — however, the on-line learning tools which actively engage students are 
currently very limited. The TQT is potentially one such tool. A general schematic view of the TQT is shown in 
[Fig. 1]. 

In the Interactive Anatomy project the TQT will be used to: 

• develop the ability to analyse clinical problems by utilising students’ knowledge of anatomical structures, 

• link procedural knowledge with anatomical knowledge, and 

• promote active learning. 

In addition the TQT can be utilised in this project and other academic disciplines to: 

• provide a mechanism for self-assessment, 

• develop procedural knowledge, and 

• facilitate the development of descriptive terminology. 

There are a number of components in the TQT environment They include: 

• a field into which the student can type an answer to a question, 

• the hints assigned by the lecturer, 

• the spelling helper, 

• the expert answers, and 

• the facility to print a record of a students’ responses and self-assessment to disk for later review (by the 
student). 
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Student Learning 



Learning is the way in which an individual changes the way s/he conceptualises the world. Teaching 
involves a lecturer constructing learning opportunities for students. One of the major difficulties in the design of 
computer-facilitated learning (CFL) is the gulf between the instructional or educational design of CFL and what 
the research literature indicates is good teaching practice [Ramsden 1992]. There is considerable evidence that 
students need to be actively engaged in knowledge construction through a variety of learning experiences 
[Laurillard 1993]. The need to make learning environments as rich as possible to enhance students’ ability to 
construct knowledge and resolve conceptual difficulties is fundamental if courseware is to be a significant part of 
course delivery in the next 10 years. Only limited examples exist where CFL has replaced part of the traditional 
approach to academic learning in higher education. At present there are very few learning tools of the type being 
developed by the MEU (Learning Engines, [Fritze, & McTigue 1997]) available for lecturers to construct 
learning opportunities which actively engage students. The self-assessment component of the TQT is one valid 
method of determining the ‘distance’ between the student response and the expert answers provided by the 
content expert. 

The TQT is not intended to: 

• supplant conventional examinations (formal assessment) although it may be possible to do so in the future, 

• address syntax, grammar or content other than in a very fundamental way (key words), or 

• address open-ended essay type questions. 




Figure 1: The structural view of the TQT 
A Functional View of the TQT 

A functional view of the current iteration of the TQT object is shown in [Fig. 2]. The structure of the 
authoring environment is critical as the tool is to be used by lecturers with limited computing experience. 
Design of the authoring environment is guided by work carried out as part of Learning Engines Project. A key 
component of the intended authoring environment is the ability to incorporate graphics, animations etc., as 
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sources of question objects. Currently, the functionality of the TQT is linked to a database. The lecturers have a 
proforma which sets the: 

• the key words, spelling terms, expert answers, and hints for students; and 

• structuring of questions with multiple forms of media (video and images from the Interactive Anatomy 
database). 

Question: Define a term, express an idea, or interpret some information. 




Text entry field 
for student. 



System analyses student 
input and provides initial 
response by highlighting 
key words. 



response to student 




The student has the opportunity 
to look at an expert answer, 
or try again. 



The student reflects upon 
her or his response, and saves 
the text to a file for later printing. 



Figure 2: A functional view of the TQT object 



Current Development and Evaluation 

Preliminary work on the TQT has been completed. The development of the TQT is focused on the 
Interactive Anatomy project currently nearing the end of its first of two years in the Medical Faculty. The first 8 
of 16 PhotoCD disks have been completed. Currently the design team is developing the first two modules for 
use with students in 1999. The TQT will be incorporated into interactive tutorials on CD-ROM which will 
facilitate the development of anatomical knowledge in a case-based clinical approach. Student evaluations of the 
modules will take place in early 1999. A screen capture of the prototype is shown in [Fig. 3]. 
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Is this likely to be an enlarged 
thyroid gland? 

Please explain precisely. 
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Figure 3: Prototype of the Text Question Tool 

Sample Questions and Answers from the Interactive Anatomy Project 

The Interactive Anatomy CD-ROM project is a large interactive multimedia project at the end of its first year 
which is part of the restructuring of the curriculum in the Medical Faculty. In each question the student would 
be told to type her or his answer in the text entry field to answer the question. Information in the Help section 
for the student will include: 

“These are technical questions requiring technical answers. You may get hints of keywords, check your spelling 
in the Spelling Tool or look at the expert answers after your first attempt. Once you have reached a decision, 
check your answer against the expert answer(s). You will be told how many of the key words or phrases you 
have entered which are appropriate to the question. You may try again if not satisfied or check your answer 
against the expert answer(s) provided and estimate your mark. These results can be saved to disk and printed out 
later.” 

Two example questions are shown in [Tab. 1]. 



Summary 

This project differs from the investigations into artificial intelligence and full text recognition and analysis of 
text by computers in a number of important ways. Firstly, it is not intended that the TQT would be semantically 
coherent, that is, be able to analyse a student response as an experienced teacher of the subject matter. There is 





considerable evidence (discussed earlier) that this level of complexity is not required in order to significantly 
enhance student learning. The second major difference is the assessment of student answers. The students are 
expected to assess their own answers and provide a mark or grade for their future reference. In the future it may 
be possible to construct a basic marking algorithm based upon the number of key words or phrases used; 
however, this is not perceived to be an educational requirement — students can learn without formal assessment. 
The TQT is an interactive learning tool which has the potential to enhance student learning. 



Year 


Question 


Expert answer 


Keywords 
and/or phrases 


First Year 
Anatomy 


The patient in the photograph, has a 
large lump in the neck that is 
observed to move upwards on 
swallowing. Is this likely to be an 
enlarged thyroid gland? Please 
explain precisely [Fig. 3]. 


It is in the midline and the thyroid 
gland moves on swallowing as it is in 
the pretracheal fascia which attaches 
to the oblique line of the thyroid 
cartilage (of the larynx). 


pretracheal fascia, 
oblique, 

thyroid cartilage, 

larynx, 

midline 


Second Year 
Anatomy 


Explain the difference between a 
direct inguinal hernia and an indirect 
inguinal hernia? 


A direct hernia pushes through a 
weakness in the posterior wall of the 
inguinal canal and is found medial to 
the inferior epigastre artery. An 
indirect hernia exits the abdominal 
cavity via the deep inguinal ring into 
the inguinal canal and is lateral to the 
inferior epigastric artery. 


medial, 

lateral, 

inferior epigastric 
artery, 

inguinal canal, 
posterial wall, 
deep inguinal ring 



Table 1: Example proforma for anatomy questions 
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